Deterministic and Stochastic Approaches to Supervisory Control Design for 
Networked Systems with Time- Varying Communication Delays 



Burak DemireP *, Corentin Briat'', Mikael Johansson^ 

"Automatic Control Laboratory, School of Electrical Engineering, KTH Royal Institute of Technology 
''Control Theory and Systems Biology, Department of Biosystems Science and Engineering, Swiss Federal Institute of Technology Zurich 



Abstract 

This paper proposes a supervisory control structure for networked systems with time-varying delays. The control 
structure, in which a supervisor triggers the most appropriate controller from a multi-controller unit, aims at improving 
the closed-loop performance relative to what can be obtained using a single robust controller Our analysis considers 
average dwell-time switching and is based on a novel multiple Lyapunov-Krasovskii functional. We develop stability 
conditions that can be verified by semi-definite programming, and show that the associated state feedback synthesis 
problem also can be solved using convex optimization tools. Extensions of the analysis and synthesis procedures to 
the case when the evolution of the delay mode is described by a Markov chain are also developed. Simulations on 
small and large-scale networked control systems are used to illustrate the effectiveness of our approach. 
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1. Introduction 

Networked control systems are distributed systems that use communication networks to exchange information 
between sensors, controllers and actuators ||Tl|2l- The networked control system architecture promises advantages in 
terms of increased flexibility, reduced wiring and lower maintenance costs, and is finding its way into a wide range of 
applications, from automobiles and transportation to process control and power systems, see e.g., ||2| - 

The use of a shared communication medium introduces time-varying delays and information losses which may 
deteriorate the system's performance, even to the point where the closed-loop system becomes unstable. A conser- 
vative approach is to design a robust controller that considers the worst-case delay. However, this might cause poor 
performance if the actual delay is only rarely close to its upper bound. Therefore, there is currently a renewed interest 
in adapting the control law to the delay evolution {e.g., [6 1 - 1 1 1 1). Inspired by the communication delays that we have 
experienced in appUcations, see Figure [T] we design a supervisory control scheme in the sense of lfT2ll . This control 
architecture consists of a finite number of controllers, each designed for a bounded delay variation (corresponding, 
e.g., to low, medium and high network load) and a supervisor which orchestrates the switching among them. 

The analysis of switched systems with fixed time-delays is challenging and has attracted significant attention in the 
literature, e.g., ||7][T3][14][l5l. Only recently, however, attempts to analyze switched systems with time-varying delays 
have begun to appear. Distinctively, |8| constructed multiple Lyapunov-Krasovskii functionals that guarantee closed- 
loop stabiUty under a minimum dwell-time condition for interval time- varying delays. An alternative approach to deal 
with time-varying delays is to assume that they evolve according to a Markov chain and develop conditions that ensure 
(mean-square) stability, see e.g., [16] - I.21J . The work in |16| assumed that the time delay never exceeds a sampling 
interval, modeled its evolution as a Markov process, and derived the associated LQG-optimal controller. However, this 
formulation is not able to deal with longer time-delays. The work in IfTTl proposed a discrete-time Markovian jump 
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Figure 1: The figure shows a recorded delay trace from the multi-hop wireless networking protocol used for networked control in 1241 . The 
delay exhibits distinct mode changes (here corresponding to one, two or three-hop communication) and varies around its piecewise constant mode- 
dependent mean. Similar behavior was reported by |25 |, who measured the delay of sensor data sent over a CAN bus. Their delay varied between 
10-20 ms, but increased abruptly to around 150 ms under certain network conditions. 

linear system formulation, which allows longer (but bounded) time delays, and posed the design of a mode-dependent 
controller as a non-convex optimization problem. Complementary to these discrete-time formulations, llT9l - 11211 have 
investigated the mean-square stability of continuous-time linear systems with random time delays using stochastic 
Lyapunov-Krasovskii functionals. The papers [22. 23 J have applied these techniques to networked control systems 
with random communication delays and synthesized mode-dependent controllers. 

In this paper, we analyze our proposed supervisory control structure by combining a novel multiple Lyapunov- 
Krasovskii functional with the assumption of average dwell-time switching. The average dwell-time concept, intro- 
duced in ||26l . is a natural deterministic abstraction of load changes in communication networks, where minimal or 
maximal durations for certain traffic conditions are hard to guarantee. We demonstrate that the existence of a mul- 
tiple Lyapunov-Krasovskii functional that ensures closed-loop stability under average dwell-time switching can be 
verified by solving a set of linear matrix inequalities. In addition, we show that the state feedback synthesis problem 
for the proposed supervisory control structure can also be solved via semi-definite programming. A similar analysis 
for Markovian time-delays is developed based on a slightly less powerful Lyapunov-Krasovskii functional than the 
one underpinning our deterministic analysis. Also in this case, we manage to design mode-dependent state feedback 
controllers using convex optimization. 

The organization of the paper is as follows. Section 2.1 presents the supervisory control structure and formalizes 
the relevant analysis and synthesis problems in a deterministic setting. In Section 2.2, multiple Lyapunov-Krasovskii 
functionals are constructed for establishing exponential stability of supervisory control systems under average dwell- 
time switchings. Additionally, LMI conditions that verify the existence of such a multiple Lyapunov functional are 
derived. State-feedback synthesis conditions are also given in Section 2.3. Section 3 develops a similar analysis 
framework for stochastic delays. Section 3.1 formulates switched control system problem introduced in Section 2 
as a Markovian jump linear system. Section 3.2 develops stochastic exponential mean-square stability conditions for 
the supervisory control system under stochastic delays. The corresponding state-feedback synthesis conditions are 
proposed in Section 3.3. Numerical examples are used to demonstrate the effectiveness of the proposed techniques in 
Section 4. Finally, Section 5 concludes the paper 

Notation: Throughout this paper, M" denotes the «-dimensional Euclidean space, is the set of all m x n real 
matrices, and §"_,_ denotes the cone of real symmetric positive definite matrices of dimension n. For a real square 
matrix M we define = M + where is its transpose. Additionally, '★' represents symmetric terms in 
symmetric matrices and in quadratic forms, ® denotes the Kronecker product, and M>o (M>o) is the set of nonnegative 
(positive) real numbers. Lastly, col(/i,) is the column vector with components 
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Figure 2: The general block scheme of the proposed supervisory control structure. 



2. Deterministic Switched Systems 

2.1. System Modeling 

We consider the supervisory control system in Figure 2. Here, G is the plant to be controlled, described by 

i(f) = Ax(t) + Bu(t) (1) 

where A e M."^" and B e M"^™. The network is modelled as a time-varying delay To-(/)(f) where cr : IR>o i-> A1 with 
Ai - {1, ■ . . , M} is the mode (operating condition) of the network. We assume that the delay in each mode is bounded, 

f^l ^ - '^o-(l)(t) < hail) ^ hM+l . 

The multi-controller unit uses the mode signal to select and apply the corresponding mode-dependent feedback law 

U(t) = Ka-(,)X{t - T^(,)(f))- (2) 

In this way, the closed-loop system is described by the following switched linear system with time-varying delay 

2i : i(f) = Ax(t) + Aa^,yx(t - T^(o(f)), Vf e M>o 

x(t)^<p(t), Vf € [-/!m+i,0] 



(3) 



where A^(,) = BK^a^ e M"^" and ,p(t) e C([-/im+i,0],M") is the initial function belonging to C([-/im+i, 0], M"), the 
Banach space of continuous functions defined on [-/im+i. 0]. 

Definition 2.1. The system ([3]) is exponentially stable under the switching signal cr{t) if there exist positive constants 
y and a such that tlie solution of x{t) of the system ([3]l satisfies 

II x(t) II < r II x(to) \\c e-«('-'»>, t > to 

where \\ xito) \\c= sup {|| x(tQ + 0) \\, \\ x(to + 6) ||). 

-/iM+l<9<0 

In order to guarantee exponential stability, we will put restrictions on the switching signal cr(t). Specifically, we 
will assume that the signal satisfies an average dwell-time condition in the following sense. 

Definition 2.2 (Liberzon [27 1). We denote the number of jumps of a switching signal cr on the interval (t,T) by 
N(r(T, t). Then we say that cr has the average dwell-time if there exist two positive numbers No and Ta such that 

T - t 



NAT,t)<No + 



vr > f > 0. 



The set of all switching signals satisfying the above condition is denoted by S[Ta]. 

We consider two specific problems in this section. The first is to verify that the switched linear system ([3]) is expo- 
nentially stable under average dwell-time switching. The second one is to design state feedback controllers for each 
mode such that the supervisory control system is exponentially stable with guaranteed convergence rate. 



2.2. Exponential Stability Analysis Using Multiple Lyapunov - Krasovskii Functionals 

The exponential stability of switched system (|3]l is equivalent to the existence of a scalar a e M>o such that e^'UxCOII 
converges asymptotically to zero for each cr e >S[rfl]. To characterize the rate of convergence of the system (|3]l, let us 
consider the change of variables ^(f) = e^'xit). Then we have: 

^(f) = ae"'x{l) + e^'xit) = Q'^(f) + e"'[Ax(f) + A,x(f - T,(f))] 

= {al„ + A)m + e^'^'WA,^ (f - T,(f)) , (4) 

where T,(f) e [/z,, V/ € M. Note that the asymptotic stability of Q implies that the original system is exponen- 
tially stable with decay rate a. However, this change of variables introduces a time-varying coefficient in the switched 
system model Q. Similar to [281, we can exploit the (mode-dependent) bounds on the delay and rewrite Q in a 
polytopic form. Specifically, we express the term e"*^'*'^ as a convex combination of the bounds e''''' and e°''''+i : 



Ai{t)e"''' + X2{t)e"''' 



where /li(f), h(t) e M>o and Ax{t) + A2{t) = 1, Vf € : 



V/ e M , 

>o. The delayed differential equation Q is then rewritten as 



(5) 



where A„ = (al„ + A) and A^,^ ^ p/yA/ withp^ = e«'''+i-i when T,(f) e [/i,-, V/ e M. 

We combine a novel multiple Lyapunov-Krasovskii functional with the dwell-time approach of f26l to establish 
exponential stability of the switched system (|3]l. For ease of notation, we state the theorem for the case of two delay 
modes only, but the approach extends immediately to a system with M modes. 

Theorem 2.3. There exists a finite constant such that the switched linear system Q is exponentially stable over 
S[ra\ with a given decay rate a > Q for time-varying delays T,(f) e [/z,-, V/ e {1,2) if there exist real matrices 
Pi, Qik,Rik,S ik, Tik G S"^ and Zik e M"^", V/, fee {1,2) and a constant scalar i-i > I satisfying Pi < /iPj, Qik < jiQjk, 



S ik 


< IJ^S jk 


and Tik 


< t^Tjk, 


Vi,j,ke 


{1,2) such that the LMIs 
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(8) 



where O,- = P,A. + AlP, + Y^ti (Qik + Rik - S ikl T,! = T,, - Z,i, Tq = Ta - Za, = Qn + r,i + = 
Gi2 + ^5 !2 + ZLi ^ik + '^"'^ ■^o - Ri2 + T'q /joW far all i,je{l,2}. 

Proof. Our claim follows if we can find Lyapunov-Krasovskii functionals y,(f) that guarantee decay rate a while in 
mode / and a constant > 1 such that y,(f) < /uVjit) V/,y € {1,2). Then, by ll26l Theorem 1], ^ is exponentially 
stable for every switching signal cr with average dwell-time t„ > t* 



In fj 

a 

We consider the following Lyapunov-Krasovskii functional, inspired from ||29]| . V, : M" 



V/ e M : 



/t=l 



Jt-ht I' 



1,-1 ^-'ii- i J-h+i Jt+s 



2 /,, , 



k=l 

The derivative of y,(f) along the trajectory of the system is given by 

2 , 2 



y,(0 = 2i^(t)Pm + f(t)\ J] {Qik + Rik)ht) -Y,^Ht- h)Qik^{t -hk)-J] fit - hk^i)Rik^(t - hk^i) 

k=l k=l k=\ 

+ f(t)\j]ihlSik + 6lTik)\m-J] [ hfis)Sik^(s)ds-Y, r ' 5kf{s)Tik^{s)ds . (10) 

^ j=\ k=\ k=\ -^'-I't+i 

Using Jensen's inequality QUI , the integral term f' . hkPis)S ik({s)ds in the preceding equality is bounded as 

- r hkfis)SikHs)ds<-m-^it-hk)VSik[m-^it-hk)]. (11) 

Jt-ln 



To upperbound the integral term [' 5kt^ (.s)Tik£{s)ds, we use the reciprocally convex combination from 11311 : 



- r ' 6kf{s)Tik^{s)ds = - r 

Jt-ht^i Jt- 



■I-Tk(t) ni-h 

5kf(s)Tik^(s)ds- 6kf{s)Tik^{s)ds , 



ft-hk)-ft-Tk(t)) 
^(t - Tk(t)) - ^(t - hk^l) 



Tik Zik 
* Tik 



^(t - hk) - ^(t - Tk(t)) 
ft-Tk(t))-^(t-hk+l) 



(12) 



>0 



Substituting ( [TT| ) and ( [T2| ) into ( [TO] i, we compute an upper bound for the Lyapunov functional (j9| as 



y,(f) < fit) AlPi + PiA, + Zti (Qik + Rik - Sik) + Al Zti (hlSik + 6lTik)A„ 



+2 fit) 



Pi E ■=! WA,^) + Al 2li (hlSik + SjTik) (i/OA.,,)J^(f - T,(0) 
+^T(f _ ri(t)) Z%i Ujma.,y[zli (hlSik + 6lT,k)\ Z%i (^>(f)A„,)f(f - T,(f)) 

- ZLi ^^(f - n(0)(2r,i - Zik - zl)ft - Tkit)) + 2f(t) Sikft - hk) 

- ZLi fit - hk){Qik + Sik + Tik)ft - hk) - fit - hk^i){Rik + Tik)ft - hk^i) 
+2 SLi f(t - nit)){Tik - Zik)ft - hk^x) + 2 fit - hk)Zikft - hk^i) 

+2 Zli fit - Tkit)){Tik - Zl)ft - hk) ^ ilfHt)Tmit) , 



where i/r(f) = col{f(f),^(f - Ti(f)),^(f - T2(t)),^(t - hi),^(t - 112), ^(t - /13)). Note that the time derivative of Vi(t) is 
bounded by a quadratic function in iff{t), i.e.. 



y,(f) < ^T(f)r,(f)<A(0 , 



with 



r,(0 = ^i(Or,i +^2«r,-2 

for all / e {1,2). Then, for two different modes, we form the following two matrices: 



where Tij = T] i - Z] , and 
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(13) 



k=l 



t>lY,{hlS2k + SlT2k)<p2 



(14) 



k=l 



By applying the Schur complement twice to r,y to form Fy, we arrive at the equivalent condition: 

r,(0 = Adt)rn + A2(t)Ti2 < 0, V/ e { 1, 2) . 



(15) 



As argued above, the condition is satisfied for all Ai(t) if Fi, and Fq are both negative definite. By guaranteeing that 
F,(f) < 0, we ensure that the dynamics in each fixed mode is exponentially stable with decay rate a. However, to 
guarantee stability for the switched system under the average dwell-time assumption, we also need to guarantee that 

VM<fiVj(t), V/,je{l,2) (16) 

for some fi > I. Noting that y,(f) is linear in f ,-, Qa, 7?,^, S it and Ta, ( fTS] ) is implied by the following conditions: 

Pi < iiPj , Qik < nQjk , Rik < I^Rjk , Sik < iiS jk , Tik < jiTjk 

for all /, i,k e {1,2). This concludes the proof. □ 

Remark 1. The analysis procedure extends immediately to the system with M modes. However, the LMIs grow in 
both size and number In contrast to the two-mode case, we need to check 2M LMIs (extensions of (|6]l, ^) whose 
dimensions are 2(2M + l)n X 2(2M + l)n, supplementary LMIs (e.g., and M(4M^ - 3M - 1) additional LMIs 
(e.g., Pi < l-iPj). The LMIs use M{5M + 1) matrix variables, each with n(n + l)/2 decision variables. 

Proposition 2.4. A lower bound on the average dwell-time ensuring the global stability of switched delay system Q 
is given by t° - Infi/uo where Oo is the optimal value of the convex optimization problem 



maxmiize 

Pi>0,Qii,>0, 
«,,->0,S,i->{)T,i>0 

subject to 



LMIs and 

Pi < ^iPj, Qik < pQjk,Rik < nRjk,Sik < fiSjk, Tik < fiTjk V i,j,k e {1,2) 



(17) 



Due to the presence of multiple product terms aPj and e"'''+' 'A,P, in (|6]l and (|7]i, the problem cannot be solved 
directly using semidefinite programming. However, the problem is easily seen to be quasi-convex. Hence, we can 
solve it by bisection in a. Since the decay rate a is inversely proportional to To, this solution procedure gives us a 
lower bound on the allowable average dwell-time t„. If we can guarantee that the average dwell-time between mode 
changes in the communication network is larger than this bound, then global stability of the closed-loop is guaranteed. 



2.3. State-Feedback Controller Design 

In this section, we will extend our analysis conditions to mode-dependent state feedback synthesis for the super- 



visory control structure introduced in Section 2. 1 More precisely, we consider a hnear time-invariant plant 

i(f) - Ax(t) + Bu{t) 

where the control input is a mode-dependent linear feedback of the delayed state vector, i.e. 



u(t) = Kix(t - r,(0) 



(18) 



when cr(f) - i (and hence, T,(f) e [/i,, /z, + 1)), / e Ai. The design problem is to find feedback gain matrices Ki that 
ensure closed-loop stability for all switching signals in >S[Tfl]. Clearly, this problem is closely related to the stabiUty 
analysis problem considered in Section 3, since the supervisory control structure induces a switched linear system on 
the form Q with Ao-(,) = BKu-ij). We have the following result: 



Theorem 2.5. For a given decay rate a > 0, there exists a state-feedback control of the form ( |18| l which exponentially 
stablizes system ^ over S[Ta] for time-varying delays Ti(t) € [/i,-, Vi € {1,2) if there exist real constant matrices 
Pi, Qik,Rik,S ik, Tik € §"+ V/, A: e {1,2) andXi,Zik € M"^" V/, A: e {1,2), and a constant scalar fi > 1 such that the LMIs 
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where E,i = g/i + f ,i + 5 ,1, S,-2 = g,2 + 5 ,2 + liLi ^« + ^'3 = ^,2 + fa, ^ik = T'/j - Z/t, ant/ P,- < j, Qik < l^Qjk, 
Rik < fJ^Rjh S ik < fJ^S jk and fik < Ijtjk V/, j,k e {1,2) are feasible. A stabilizing control law is given by ( fTS) with gain 
Ki = YiXr^forallie {1,2}. 

Proof: The structure of (|6| and (|7| is not suitable for the synthesis of a state-feedback controller due to the presence of 
multiple product terms AaS ik, AaT/k, Aa. S ik and Aa. Tik. These product terms prevent finding a linearizing change of 
variable even after congruence transformation. Instead, we will use the relaxation term introduced in Briat et. al \ 32| to 
decouple the products at the expense of an increased conservatism. Denote {T9\ and pO] ) by ©i^ and ©2;, respectively. 
Then we prove that 0,y < V/ € { 1 , 2) implies the feasibility of (|6]) and (|7]). Note that ©,y can be decomposed as 

&ij = ©olx,=o + UjXiVi + VjXj Ui<Q, V/ e { 1 , 2) 

where Ui ^[-I„Aa A„, , 0„x4« 4 0„x4n], Vi = [/„ 0„xii„], U2^[-I„ A„ 0„ A„y 0„x3« 4 0„x4«] and V2 = [/„ 0„xii«]- 
Then invoking the projection lemma [33] , the feasibility of 0,y < implies the feasibility of the LMls 

Nl^®ij\x,=oNu, < (22) 

N^,®ijk=QNv, < (23) 

where Nu, and Nv, are basis of the null space of Ui and V,, respectively. After some tedious calculations, we can 
show that LMls (|24]) and ( pSj l are equivalent to (|6| and ([7]l showing that ©,j < V/ e {1,2) implies the feasibility 
of (|6]l and (|7]l. Moreover, LMl ( |23] l characterizes the conservatism of the relaxation. 
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Since LMIs ( p4] i and ( p5| ) do not include any multiple product, it can easily be used for controller design. Hence, 
it is possible to use congruence transformations and change of variables so as to design the state-feedback controller. 
Performing a congruence transformation with respect to matrix /i2„ ® X"' and applying the following linearizing 
change of valuables 1,- ^ X7\ ^ XjPiXi, Q,, ^ Xj Qi.Xi, Rik = XjRi.X,, 5,-, ^ 1/5, f« = l^r,*!,-, S,i ^ 
XjEaiti, Ea = XjEaXi, H,3 = I^Hb^/, Z,* = l/Z;^!,- and = V/, k e {1,2} yields LMI ([T9| and □ 

Remark 2. The synthesis procedure readily extends to systems with M modes, yet both the size and the number of 
LMIs increase. Specifically, we need to check 2M LMIs (extensions of \19\ , ( |20| lj whose dimensions are 2(2M + 2)n X 
2(2M + 2)n and M{5M^ - 3M - 1) additional LMIs (e.g., (|2TJ, Pi < f^Pj). In total, these LMIs comprise M{5M + 3) 
matrix variables, each of which has n(n + l)/2 decision variables. 



3. Stochastic Switched Systems 

Our deterministic modeling framework has several advantages: it allows to model long time-delays, is able to 
account for mode-dependent delay bounds and admits a convex formulation of the (mode-dependent) state feedback 
synthesis problem. However, it also has a disadvantage in that it does not allow to account for more detailed knowledge 
about the evolution of the delay mode beyond the average dwell-time. It is therefore interesting to derive similar results 
when the delay mode varies according to a Markov chain, cf. |fT9l l20l 1211 [34i . Such results will be developed next. 

3.1. System Model 

Let us consider a dynamical system in a probability space (Q, ;F, P), where Q. is the sample space, 'F is the cr- 
algebra of subsets of the sample space and P is the probability measure on 'F. Over this probability space, we consider 
the following class of linear stochastic systems with Markovian jump parameters and mode-dependent time delays 

1.2 : i(f) = Ax(t) + Ar(,)x{t - T,(o(f)) , Vf e M>o , 

Here, x(t) e W is the state, A e M"^" and A,.(,) e M"^" are the known system matrices while {r,,f e IR>o) is a 
homogeneous, finite-state Markovian process with right continuous trajectories and taking values in the finite set 
Ai - {I,-- - ,M). The Markov process describes the switching between the different modes and its evolution is 
governed by the following transition probabilities 



P[rt+A ^ j\r, = i] = 



TTijA + o(A) if i + j 
1 + TT/jA + o(A), if i - j 



where nij is the transition rate from mode / to j with 7r,j > when / /' and na - - j^- 7r,y and o(A) is such that 
limA^o = 0. Furthermore, Tr(t){t) is the time-varying stochastic delay function satisfying 

hi < h^^^^ < Tr(,){t) < hrd) < hM+\ . 

Finally, (p{t) is a vector-valued initial continuous function defined on the interval [-/jm+iO], and ro e M are the initial 
conditions of the continuous state and the mode. 



Definition 3.1. The Markovian jump system \26\ is exponentially mean-square stable if there exist positive constants 
a and y such that 

E[ II x(t) f I <fi(to),rr„] < y II x(fo,r,„) e''^^'-"'^ 
holds for any finite (p(to) e M" defined on [-/im+i, 0] and any initial mode r,g e M. 
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3.2. Exponential Stability Analysis Using Stochastic Lyapunov-Kmsovskii Functionals 

In this subsection, we analyze the exponential stability of the Markovian jump linear system ([26| using a similar 
approach to what we developed for the switched delay case. To portray the convergence rate of the system ( |26] l, we 
thus consider the change of variables ^(t) ^ e"'x{t) and find 



^(f) = (al„ + A)^(f) + e"^'<"«A,(,)af - T,(,)(f)) 
where Tr(() 6 [/!^(,), /zr(o)- For ^^ch r(t) - i, V/ € M, we rewrite ( [27] i as 

^(f) = (a/„ + A)f(f) + e«^'»A,f(f - r,(f)) ■ 



(27) 



(28) 



Using the same polytopic approach as in Section 2, we express e"'^'^'^ as a convex combination of its mode-dependent 
bounds: 



Ai{t)e"''' + A2it)e 



MieM 



where A\{t),A2{t) e IR>o and A\(t) + Aiit) = 1, Vf e ]R>o. Thus, the stochastic switched system ( [26| l can be defined, 
for each r(t) - i, V/ e Ai, as 



m = A,^(f) + 2 /i/f)A,,/(f - T,(f)) 

where A„ = (a/n + A) and A„,^ ^ e,jA/ with gij = £'°''''+j-i when T,(f) e [hi, /z,+i) , V/ e M. 



(29) 



Theorem 3.2. The Markovian jump linear system ( |26| l exponentially mean-square stable with a given decay rate 
a > for randomly varying delays t,- e [/z,-, /z,+i), V/ e {1,2) if there exist matrices Pi,Qi,Ri e S" + , V/ e {1,2), 
S,T,Q,'R^ §"+ andZ € M"^" iMc/i f/zaf the following LMIs hold for ij 6 {1,2) 
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< , 



where (i>i ^ AlPi+PiA,,+Z%i TTijP j+{Qi+h2Q+6iQ,)+{Ri+hi'R+62R,)-S , eij = h^W^ ande2j ^ 6'r+r]6^.,^'^ 
with rj = max Itt,-,!, k = argmax|7r,;| and (S^ax = max - /!,|, V/ € {1,2), and T - T - Z. 

Proof. We define a stochastic Lyapunov-Krasovskii functional V :M." x M 

5 



(30) 



(31) 



M>o candidate for the system p6| ) as 
V(^„ r,) = ^^(t)P,^„^(t) + ^ Vk(^„ r,) , (32) 



Vi(ft,r,) 



k=2 
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where 



Jt-h,,,\ 



■0 



f {0)Q,^{e)dBd. 

hi Jt+s 



-hi Jl+s 



hi 

-hi r' 



/i.,„ Jf+j J-Zi. Js Jt+e 



^''ie)R,^(e)deds 

hi Jt+s 



■0 ,-0 r-t 



+ T]6l 



^''(v)S^(v)dvd0ds 



-hi Js Jt+6 

-'i,.„, rt 



V5(f„r,)= (V) + I f{v)T^{v)dvdeds 

^ ' J-h,.„f Jl+s J-hi Js Jt+9 



with 77 = max \nii\, k = argmax|;7r,-,|, and d^ax — max - V/ € Ai. 

Definition 3.3 (0ksendal |35|). The weak infinitesimal operator J{ of the Markovian process {{x{t),rt),t > 0} is 
defined by 



:RV(x(t), r,) = Hm 

A^O 



E[y(x(f + A), r,^^)\r] - V{x{t), r,) 



where Tt - o-{{x{t), r,),t> 0). 

Straightforward but tedious calculations yield that, for each r, = /, / e M, along solutions of (p6|l, we have 



1 

^Vl = ^T(f) AlPi + PiA„ + ^ TlijPj ^(f) + 2^T(f)p. .(f)A„,^)^(f - T,(f)) 

I- i=l J ./=1 

2 „/ 
;-i Jt-h, 



(33) 



r + ^ r ' ^iis)Q,^{s)ds 

Jt-hi J t-h^ 



(34) 



2 „t 

j=i Ji-hj+i 



{s)Rj^(s)ds + hi^Ht)mt) + 62Tl^Ht)R,^{t) 



-t-hi 



r ^Hs)n{s)ds + r, r ^Hs)R,^{s)ds 

Jt-hi Jt-hi 



(35) 



= /i ■ (f)5 ^(f ) - hi I ^(^s)cls + ^ 7r,7 /I; I I (0)5 ^(e)^^^^^ 



/i-^ - h^ 
+ il^Y^^'{t)S^(t)-r, 

'-t-hi 



X-I'i ri 
hn J/4 



h2 



■0 



(0)8 ^{e)d0ds + {h2 - hi) 



^''i0)S^{0)d0ds 



hi Ji+s 



(36) 



Jt-h,., J -hi., Jt+s 



hj - hi 



^Ht)T^it) - 6,^,x f ' f ^H0)Tmd0ds 

J -hi Jt+s 



(37) 
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Similar to Section 2.2 
the preceding equalities as 



_^ hi^'^{s)S^{s)ds and J^_^ Si^'^ {s)T^{s)ds that appear in 

follows: 



Jt-h, 

-X 



^(f-T,(f))-^(f-/z/+l) 



T Z 
★ T 



at - Tl(t)) - - hl^l) 



(38) 
(39) 



where [It] - holds. 

In addition, for the stochastic formulation, we need to upper bound a number of additional integrals. We do so by 
noting TTij > for j + i and tt,,- < 0, and that 

M „t M „t r-t 

y^,7 ^''{s)Qjas)ds^ynij \ f(s)Qjas)ds + ^^(s)Q,as)ds 

Jl-hu \ :+! I Jt-hi 



(40) 



= r ^Hs)[Q-niiQ]as)ds +n, f ^Hs)Qias)ds 
< r ^Hs)Qas)ds +n [ ' ^Hs)Q,as)ds 

•Jt-I'iu Jt-I'iu 

where Yjf=i ^ijQj ^ <3- A similar upper bound is readily established for Yjf=i ^ij J,'_i, ^^(s)Ras)ds. We also bound 



y_[ J-hj Ji+s j^j J-hj Jr+ 



-hj Jl+s 
.0 ri 



^''(e)s^(e)deds 



-hi Jl+s 
.0 



-h\ Jt+s 



and 



< -JTiihM f f f{0)S^{e)deds + jtiJi 
= -yr„hM r ' r f{e)Smd0ds-n„{hM-h) 
<rihM f ' r emmdeds +rj{h^-h) 



Yj7Tijihj+i - hj) f ' r {9)T x(0)deds < ri6^^^ f f {0)T x{ff)d6ds . (42) 

■1 J-hi+i Jt+s J -h.M+\ *Jt+s 



^''(e)s^(e)deds 



(41) 



'-111 Jl+s 

-.0 rt 



^He)s^{e)deds 
f{e)s^{e)deds 



Now, substituting ([38]l - ( |42] i into p3| ) - ( [37) l, we get the following inequality 



AlPi + PiA„ + ^ TiijPj + {Qi + h2Q + 6iQ,) + {Ri + + 62R.) -S + Aliens + e2,,T)A„ 



Pi + AlieijS + £2jT) 



^(^/f)A„,)^(f - T,(f)) - fit - hdiQi + S+ T)at - hi) + 2^Ht)Sat - hd 
- ^Ht - hM){Ri + T)at - hM) + - T,(0)( -2T+Z + Z^)at - T,(0) + 2^T(f _ r.^t)){T - Z^)^! - K) 

2 2 
+ f{t- T,(f)) ^(^/f)A<,,pT(^j .5 + Y}Aj(t)A,^)x{t - T,(f)) 



= iA^(f)f,(f)<A(f) > 



(43) 
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where i/r(f) - col{^(f), ^(f - T;(f)), ^(f - ^(f - Note that ^V{^,, r,) is bounded by a quadratic function in i/'(f): 

(44) 

where 



:RV(s„i)<if,\t)TMt), 

fij^A,{t)fii+A2{t)fi2 



for all i e {1,2) and 



S 



★ ,-2r +Z + ZT, r-zT 





r -z 



★ 
★ 



-{Qi + S+T) Z 



(45) 



(46) 



where = [ A Aq,,^ 0„x2« ] for all /, j e {1,2). As in Section 2, applying the Schur complement lemma to r,y to form 
Yjj, we arrive at the equivalent condition 



r,(f) = A,{t)Tii + A2(t)ra < , V/ e { 1, 2) . 



(47) 



This condition is satisfied for all /I, if Tn and r,2 are both negative definite. Hence, satisfying the condition Yjj < 0, 
we guarantee that the dynamics is exponentially stable with decay rate a. This concludes the proof □ 

Remark 3. In order to investigate the stability of the Markovian jump system \26) with M modes, we must examine 
2M LMIs (e.g., ( |31[ l) whose dimensions are 6n X 6n and, additionally, 2M + 1 small size LMIs (e.g., In total, 

we use 3M + 5 matrix variables, each with n(n + l)/2 decision variables. 

3.3. State-Feedback Controller Design 

Hereafter we concentrate our interest on extending analysis conditions to state-feedback synthesis for exponential 
mean-square stability of the Markovian jump linear system scheme introduced in Section [3T| To this end, we consider 
a linear time-invariant plant 

i(f) = Ax(t) + Bu(t) (48) 

where x{t) e M." is the state and u{t) e M™ is the control input being mode-dependent linear feedback of the delayed 
state with the following control law 

u{t) = Kix(t - T,(f)) (49) 

when r(t) - i (and hence, T,(f) e [/i,, /!,+i)), / e Ai. Indeed, the design problem is to determine a set of state- 
feedback gain matrices Kt that guarantees the closed-loop control stability in exponentially mean-squared sense for 
the transition rates 11 = [7rij]ij=i, - ,m- This problem is closely related to the stability analysis problem discussed in 
Section [3^ bee ause the control system structure can also be represented as a Markovian jump linear system on the 
form of p6[) with A^(,) = BKr{i). We have the following result. 

Theorem 3.4. For a given decay rate a > Q, there exists a state-feedback control of the form \A9) that stabilizes 
system \26) for randomly varying delays t; e [hi, hi+i), V/ e {1,2) in exponentially mean-squared sense if there exist 
real constant matrices Pi, Qi, Ri 6 S"+, Vi G { 1, 2), 5 , f , (3, ^ G §"+ and X,Z € M"^" such that the following LMIs 
hold for i, j G {1,2) 

2 2 

>0, ;^7r„e,<Q, ^'^'V^i^^' (50) 
and 



T Z 
ZT T 
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(51) 



where 3,- = ^ijPj + (Qi + hiQ + 6i Q,) + (Ri + h^li + 62R.) - Pi - S , e,j = + jj'-^ and ej,; ^5]+ t]S^,, ^ 
with rj ^ max \nii\, k = argmax|7r,i| and Jmax = max - V/ e {1,2), and T - T - Z. A stabilizing control law is 
given by (|49| with gain Ki - for all / e { 1, 2). 



Proof. The proof of Theorem |3.4| is similar to that of Theorem 2.5 but an outline of the proof is included for complete- 
ness. Again, the structure of ( [3T] l is not adapted to the controller design due to the existance of the multiple product 
terms A„5, A„r, Aq._ 5 and Aq,, T preventing to find a linearizing change of variable even after congruence transfor- 



mations. As a result, a relaxation approach is applied (as in § 2.3 1 to remove the multiple product terms preventing 
the change of variables. We let ( |5T| be called ^P/y and we prove the condition ^'ij < 0. Similarly to 0,j (but with the 
difference that we have X instead of X,), ^'ij can be decomposed as follows: 

*P,7 = *P,7lx=o + UjXVi + VjX^ Ui<0, V/ e { 1 , 2) 

where f/, = [ - /„ Aa^^ 0„x2n 4 0„x2n] and V, = [/„ 0„x7n]- Then invoking the projection lemma Il33l . the feasibility 
of ^ij < implies the feasibility of the LMls 

<,,*P,-,lx=o^f/, < , 
Nl'i'ij\x=oNv, < , 
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where Nu. and Nv^ are basis of the null space of t/,- and V,, respectively. Subsequently, the inequality 
as 

"XTA^H-P/XTA^,,! 

^^lll^llllA 

I * -(Ri + T) 
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is obtained 
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<0 



(52) 



We substitute the closed-loop system ( |48] l into the inequality ( (3T| , and introduce a constant matrix X € M"^". 
Then, we compel X to be constant, and we perform a congruence transformation with respect to matrix /g,, i8> X"' 
and apply the following Unearizing change of variables 1 = X ', P, ^ X''PiX, Qi = X''QiX, Q = X''QX, Ri = 
X'^RiX, = X'^'RX, S ^ X'^SX, T = X'^TX, En = X'^EnX, Sq = Its^I, E,3 = X'^E^X, Z = X'^ZX and 
Yi = KiX, V/ e { 1, 2) in (|52|, LMl ([STJ is derived. □ 

Remark 4. To design a set of stabilizing controllers for Markovian jump system ( as discussed in § \3.3\ with M modes, 
2M LMIs fe.g., ( |51| l) whose dimensions are 8n X 8n, and also 2M + 1 small size LMIs (e.g., \5Ql\ ) must be checked. 
All these LMIs include 3M + 6 matrix variables with n(n + l)/2 decision variables. 
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Figure 3: The left figures illustrate a sample evolution of the time delay and the associated delay bounds that define the two controller modes; the 
associated mode evolution is shown in the bottom left figure. The right figure shows a representative state trajectory of the closed-loop system 
under supervisory control (dashed line) and mode-independent state-feedback (solid). Both simulations are performed from the same initial value. 



4. Numerical Examples 



We are now ready to demonstrate the proposed technique on numerical examples. Our first example, a simple DC- 
motor model, is included to demonstrate the flexibility of our control structure compared to a single robust controller. 
The second example is taken from wide-area power systems, and demonstrates that the numerical techniques scale to 
non-trivial system dimensions. Finally, we return to the small-scale example to illustrate the analysis procedures for 
the random Markovian delay model. 

4.1. Small-scale Example: DC Motor 
Consider the following linear system 



-10 



x(t) + 





0.024 



u{t) 



(53) 



with a time-varying communication delay between sensor and controller that behaves as shown in Figure [3] The 
supervisor generates the switching signals shown in Figure[3]and triggers the most appropriate controller according to 



u{t) 



KLx{t - TL(t)) if TL(t) e [20, 70) ms , 
KMx{t - Tmit)) if Tmit) G [70, 200) ms , 
KHx{t - TH(t)) if Tnit) e [200, 300) ms . 



(54) 



By solving the optimization problem ( [T7] i for = 1 .4, we find the lower bound on the average dwell-time t° - 
0.12s and the corresponding convergence rate Co = 2.78 guaranteed for the gains 





[-1421.0 


-138.9 


Km - 


[-1035.9 


-101.5 


Kh = 


[-757.09 


-72.71 



Using the same class of Lyapunov-Krasovskii functionals, we design a classical state-feedback controller to compare 
the non-switching and switching control performance. Specifically, we use the Lyapunov-Krasovskii functional (|9]l 
wifli Qk = 0, /^t = 0, 5a = 0, = 0, Vfc > 1 and T(r) e [20, 300) ms, and find o- = 1.72 for the state feedback gain 

K = [-765.74 -75.74] . 
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Figure 4: IEEE nine-bus power system. 



We observe that the switching controller has a better performance than the non-switching one, o-o - 2.78 > o- = 1.72, 
and that the controller in the low-delay mode can be made much more agressive when we use a mode-dependent 
controller The improved convergence rate is confirmed by the simulations shown in Figure |3] 

For diff'erent values of /z2 and /13, which define the boundary between the low-, medium- and high-delay mode, 
we compute the exponential decay rate a of the supervisory controller, and then try to find the largest /i^ax for which 
a single mode-independent controller can guarantee the same decay rate. The results are depicted in Table [T] As 
can be seen, increasing /j2 and decreasing /13 allow to guarantee an improved decay rate of the switching controller. 
To guarantee the same decay rate under a mode-indenendent controller, the maximum delay must be reduced, and 
sometimes significantly so. 



Supervisory Control Non-Switching Control 



a 


hi 


hi 




/14 






3.00 


20 


100 


200 


300 


20 


158 


2.42 


20 


100 


250 


300 


20 


204 


2.78 


20 


70 


200 


300 


20 


173 


2.27 


20 


70 


250 


300 


20 


219 



Table 1: Comparision switching vs. non-switching controllers for different decay rates and delay intervals 



4.2. Large Scale Example: Wide-Area Power Networks 

To demonstrate the applicability of our methods to systems of higher dimension, we consider the IEEE nine-bus 
system [36 1 shown in Figure]?] We adopt a second order (swing) model with phase and frequency (5,-, w,) for all 
generators and use the Power System Analysis Toolbox 1 37 1 to obtain the following numerical model 
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u(t). (55) 



16 




supervisoiy controller 
single controller 



5 10 15 5 10 15 

Time [s] Time [s] 

Figure 5: The left figures show a sample delay evolution and the delay bounds that define the supervisory control modes (top) along with the 
associated mode evolution (bottom). The right figure illustrates a representative state trajectory of the closed-loop system under supervisory control 
(dashed line) and a single mode-independent state-feedback (solid). Both simulations are performed from the same initial value. 



We assume that the phase and frequency of each bus can be measured and be communicated to a central controller. 
In wide-area power systems, the communication delays vary depending on communication technologies, protocols and 
network load. In this example, we assume that the delay varies between 20 and 1 10 ms (see, Figure|5]l and that mode 
changes are such that the average dwell-time is guaranteed to be at least 0.35 seconds. 

To begin, the supervisory controller is designed for the delay intervals [20, 50) and [50,110) ms. We design 
supervisory control gains targeting a decay rate of a = 0.9, and use /i = 1.4 to guarantee Ta - 0.3739. As a result, we 
compute the following control gain matrices 

Kl = [-181.60 -53.94 -83.97 -2090.02 -647.79 -85.90] , 

Kh = [-159.82 -49.22 -48.09 -1714.38 -547.06 -94.92] . 

Solving the LMIs for this larger system takes a few hours, but appears to be numerically stable. 

It turns out that an exponential decay rate of ff = 0.9 can also be achieved by the mode-independent feedback gain 

[-153.71 -48.22 -41.69 -1605.33 -514.67 -96.07], (56) 

which is less aggressive than the controller used in the low-delay mode of the switching controller As depicted 
in Figure [5] simulations are carried out for the delay trace and corresponding switching singnals shown in the left 
of the same figure, the switching controller damps oscillations between generators Gi and G3 better than its mode- 
independent counterpart. 

4.3. Markovian Jump Linear System Formulation 

Finally, we return to the DC-motor example to illustrate the applicability of the analysis tools developed in Sec- 
tion[3]for random delays. We consider the system ( |53] l with low traffic delays {i.e., € [20, 70)ms) and high traflic 
delays {i.e., th e [70, 300)ms). The switching between these two modes is described by the following transition 
probability rate matrix: 

'-P P 
q -q 

whose invariant distribution is n'^ - q/{p + q) and tt^ = pl{p + q). We vary the transition probabilities p and q, 
compute mode-dependent controllers and the rate of exponential mean-square stability. The results are summerized 
in Table |2] While mitigating max {p, q], the decay rate is increasing and controllers are getting more aggressive. 
Furthermore, when tt^ decreases, we also observe a slight decrease in the decay rate. 
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1.45 


50.00 


50.00 


1.5 


1.5 


[-810.53 


-80.11] 


[-637.13 - 


62.95] 



Table 2: Results of Markovian jump linear controller design for /ji = 20ms, h2 = 70ms and = 300ms 



5. Conclusion 

This paper has been dedicated to supervisory control of networked control systems with time- varying delays. 
The main contribution of this paper is to develop a stabiUty analysis and a state feedback synthesis technique for a 
supervisory control system that switches among a multi-controller unit based on the current network state. A novel 
class of Lyapunov-Krasovskii functional were introduced that, somewhat remarkably, admit both the analysis and 
state feedback synthesis problems to be solved via convex optimization over linear matrix inequaUties. In addition, 
we also investigated the corresponding problem for a class of stochastic systems with interval-bounded time-varying 
delay. Sufficient conditions were estabhshed without ignoring any terms in the weak infinitesimal operator of the Lya- 
punov Krasovskii fimctional by considering the relationship among the time- varying delay, its upper bound, and their 
difl'erence. Finally, examples were given to show the efl'ectiveness of the proposed analysis and synthesis techniques. 
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